Machine-learning assisted scheduling optimization and its application in quantum chemical calculations.

Easy and effective usage of computational resources is crucial for scientific calculations, both from the perspectives of timeliness and economic efficiency. This work proposes a bi-level optimization framework to optimize the computational sequences. Machine-learning (ML) assisted static load-balancing, and different dynamic load-balancing algorithms can be integrated. Consequently, the computational and scheduling engine of the ParaEngine is developed to invoke optimized quantum chemical (QC) calculations. Illustrated benchmark calculations include high-throughput drug suit, solvent model, P38 protein, and SARS-CoV-2 systems. The results show that the usage rate of given computational resources for high throughput and large-scale fragmentation QC calculations can primarily profit, and faster accomplishing computational tasks can be expected when employing high-performance computing (HPC) clusters.

RAPTOR: Ravenous Throughput Computing

We describe the design, implementation and performance of the RADICAL-Pilot task overlay (RAPTOR). RAPTOR enables the execution of heterogeneous tasks-i.e., functions and executables with arbitrary duration-on HPC platforms, pro-viding high throughput and high resource utilization. RAPTOR supports the high throughput virtual screening requirements of DOE's National Virtual Biotechnology Laboratory effort to find therapeutic solutions for COVID-19. RAPTOR has been used on 8300 compute nodes to sustain 144M/hour docking hits, and to screen 1011 ligands. To the best of our knowledge, both the throughput rate and aggregated number of executed tasks are a factor of two greater than previously reported in literature. RAPTOR represents important progress towards improvement of computational drug discovery, in terms of size of libraries screened, and for the possibility of generating training data fast enough to serve the last generation of docking surrogate models. © 2022 IEEE.